{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Express sklearn pipeline as codeflare pipeline\n",
    "Reference: https://scikit-learn.org/stable/auto_examples/kernel_approximation/plot_scalable_poly_kernels.html#sphx-glr-auto-examples-kernel-approximation-plot-scalable-poly-kernels-py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Scalable learning with polynomial kernel aproximation\n",
    "\n",
    "This example illustrates the use of :class:`PolynomialCountSketch` to\n",
    "efficiently generate polynomial kernel feature-space approximations.\n",
    "This is used to train linear classifiers that approximate the accuracy\n",
    "of kernelized ones.\n",
    "\n",
    ".. currentmodule:: sklearn.kernel_approximation\n",
    "\n",
    "We use the Covtype dataset [2], trying to reproduce the experiments on the\n",
    "original paper of Tensor Sketch [1], i.e. the algorithm implemented by\n",
    ":class:`PolynomialCountSketch`.\n",
    "\n",
    "First, we compute the accuracy of a linear classifier on the original\n",
    "features. Then, we train linear classifiers on different numbers of\n",
    "features (`n_components`) generated by :class:`PolynomialCountSketch`,\n",
    "approximating the accuracy of a kernelized classifier in a scalable manner.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Automatically created module for IPython interactive environment\n"
     ]
    }
   ],
   "source": [
    "print(__doc__)\n",
    "\n",
    "# Author: Daniel Lopez-Sanchez <lope@usal.es>\n",
    "# License: BSD 3 clause\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import fetch_covtype\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import MinMaxScaler, Normalizer\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.kernel_approximation import PolynomialCountSketch\n",
    "from sklearn.pipeline import Pipeline, make_pipeline\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the Covtype dataset, which contains 581,012 samples\n",
    "with 54 features each, distributed among 6 classes. The goal of this dataset\n",
    "is to predict forest cover type from cartographic variables only\n",
    "(no remotely sensed data). After loading, we transform it into a binary\n",
    "classification problem to match the version of the dataset in the\n",
    "LIBSVM webpage [2], which was the one used in [1].\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = fetch_covtype(return_X_y=True)\n",
    "\n",
    "y[y != 2] = 0\n",
    "y[y == 2] = 1  # We will try to separate class 2 from the other 6 classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we select 5,000 samples for training and 10,000 for testing.\n",
    "To actually reproduce the results in the original Tensor Sketch paper,\n",
    "select 100,000 for training.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=5_000,\n",
    "                                                    test_size=10_000,\n",
    "                                                    random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now scale features to the range [0, 1] to match the format of the dataset in\n",
    "the LIBSVM webpage, and then normalize to unit length as done in the\n",
    "original Tensor Sketch paper [1].\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm = make_pipeline(MinMaxScaler(), Normalizer())\n",
    "X_train = mm.fit_transform(X_train)\n",
    "X_test = mm.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a baseline, train a linear SVM on the original features and print the\n",
    "accuracy. We also measure and store accuracies and training times to\n",
    "plot them latter.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVM score on raw features: 75.62%\n"
     ]
    }
   ],
   "source": [
    "results = {}\n",
    "\n",
    "lsvm = LinearSVC()\n",
    "start = time.time()\n",
    "lsvm.fit(X_train, y_train)\n",
    "lsvm_time = time.time() - start\n",
    "lsvm_score = 100 * lsvm.score(X_test, y_test)\n",
    "\n",
    "results[\"LSVM\"] = {\"time\": lsvm_time, \"score\": lsvm_score}\n",
    "print(f\"Linear SVM score on raw features: {lsvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we train linear SVMs on the features generated by\n",
    ":class:`PolynomialCountSketch` with different values for `n_components`,\n",
    "showing that these kernel feature approximations improve the accuracy\n",
    "of linear classification. In typical application scenarios, `n_components`\n",
    "should be larger than the number of features in the input representation\n",
    "in order to achieve an improvement with respect to linear classification.\n",
    "As a rule of thumb, the optimum of evaluation score / run time cost is\n",
    "typically achieved at around `n_components` = 10 * `n_features`, though this\n",
    "might depend on the specific dataset being handled. Note that, since the\n",
    "original samples have 54 features, the explicit feature map of the\n",
    "polynomial kernel of degree four would have approximately 8.5 million\n",
    "features (precisely, 54^4). Thanks to :class:`PolynomialCountSketch`, we can\n",
    "condense most of the discriminative information of that feature space into a\n",
    "much more compact representation. We repeat the experiment 5 times to\n",
    "compensate for the stochastic nature of :class:`PolynomialCountSketch`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVM score on 250 PolynomialCountSketch features: 76.11%\n",
      "Linear SVM score on 500 PolynomialCountSketch features: 77.44%\n",
      "Linear SVM score on 1000 PolynomialCountSketch features: 77.87%\n",
      "Linear SVM score on 2000 PolynomialCountSketch features: 78.03%\n"
     ]
    }
   ],
   "source": [
    "n_runs = 3\n",
    "for n_components in [250, 500, 1000, 2000]:\n",
    "\n",
    "    ps_lsvm_time = 0\n",
    "    ps_lsvm_score = 0\n",
    "    for _ in range(n_runs):\n",
    "\n",
    "        pipeline = Pipeline(steps=[(\"kernel_approximator\",\n",
    "                                    PolynomialCountSketch(\n",
    "                                        n_components=n_components,\n",
    "                                        degree=4)),\n",
    "                                   (\"linear_classifier\", LinearSVC())])\n",
    "\n",
    "        start = time.time()\n",
    "        pipeline.fit(X_train, y_train)\n",
    "        ps_lsvm_time += time.time() - start\n",
    "        ps_lsvm_score += 100 * pipeline.score(X_test, y_test)\n",
    "\n",
    "    ps_lsvm_time /= n_runs\n",
    "    ps_lsvm_score /= n_runs\n",
    "\n",
    "    results[f\"LSVM + PS({n_components})\"] = {\n",
    "        \"time\": ps_lsvm_time, \"score\": ps_lsvm_score\n",
    "    }\n",
    "    print(f\"Linear SVM score on {n_components} PolynomialCountSketch \" +\n",
    "          f\"features: {ps_lsvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train a kernelized SVM to see how well :class:`PolynomialCountSketch`\n",
    "is approximating the performance of the kernel. This, of course, may take\n",
    "some time, as the SVC class has a relatively poor scalability. This is the\n",
    "reason why kernel approximators are so useful:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kernel-SVM score on raw featrues: 79.78%\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "ksvm = SVC(C=500., kernel=\"poly\", degree=4, coef0=0, gamma=1.)\n",
    "\n",
    "start = time.time()\n",
    "ksvm.fit(X_train, y_train)\n",
    "ksvm_time = time.time() - start\n",
    "ksvm_score = 100 * ksvm.score(X_test, y_test)\n",
    "\n",
    "results[\"KSVM\"] = {\"time\": ksvm_time, \"score\": ksvm_score}\n",
    "print(f\"Kernel-SVM score on raw featrues: {ksvm_score:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, plot the resuts of the different methods against their training\n",
    "times. As we can see, the kernelized SVM achieves a higher accuracy,\n",
    "but its training time is much larger and, most importantly, will grow\n",
    "much faster if the number of training samples increases.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAGpCAYAAADhiRM+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA3RElEQVR4nO3deXxU5fn//9eVgCAWxAWVGmUJhDUhrIIWorJYEFEqspSq2AKyWBH7o+VXP1pba62WTytaPyJWRZEK4l5QWqEgKFoWTRRkCYGoCWgjioIsSnJ9/5jJkIQkJJDJcvJ+Ph7zyMw9Z7nmMPDmvs/JfczdERERCYqYqi5ARESkIinYREQkUBRsIiISKAo2EREJFAWbiIgESp2qLqAszjzzTG/evHlVlyEiItXI+vXrP3f3JkXba0SwNW/enHXr1lV1GSIiUo2Y2UfFtWsoUkREAkXBJiIigaJgExGRQKkR59iK891335GVlcXBgweruhSRWqV+/frExcVRt27dqi5FpFg1NtiysrJo2LAhzZs3x8yquhyRWsHd2b17N1lZWbRo0aKqyxEpVo0dijx48CBnnHGGQk2kEpkZZ5xxhkZKpFqrscEGKNREqoD+3kl1V6ODTUREpCgF2wn43ve+d1TbrFmzeOqppyq1jkWLFtG5c2c6depE+/bteeSRR1ixYgW9evUqtNzhw4c5++yz2bVrF2PGjKFBgwbs3bs38v6UKVMwMz7//PNKrV9EpCLV2ItHqqsJEyZEdfvujrsTExP6P8l3333H+PHjWbNmDXFxcRw6dIjMzExat25NVlYWmZmZ5E9HtnTpUjp27EjTpk0BaNWqFS+//DI/+clPyMvLY/ny5Zx77rlRrV9EJNpqVY9t195dxD8Qz6f7Po3aPu68805mzJgBwMUXX8yvfvUrevToQUJCAqtWrQIgNzeXadOm0b17d5KSknjkkUcA2LdvH3379qVLly4kJiby8ssvA5CZmUm7du2YNGkSXbp04ZNPPonsb+/evRw+fJgzzjgDgHr16tGmTRtiYmK45pprWLBgQWTZ+fPnM2rUqMjrUaNGRd5fsWIFF110EXXq6P86IlKz1apgu2vlXWTuyeSuN+6qtH0ePnyYNWvWcP/99/Pb3/4WgMcee4xTTz2VtWvXsnbtWh599FF27NhB/fr1efHFF3n33XdZvnw5v/jFL3B3ALZs2cJ1113He++9R7NmzSLbP/300xkyZAjNmjVj1KhRzJs3j7y8PCAUXPPnzwfg0KFDvPrqq1x99dWRdVu3bk1OTg5ffvklzzzzDCNHjqyswyIiEjW1Jth27d3FE6lPkOd5PJH6RFR7bQX96Ec/AqBr165kZmYC8K9//YunnnqK5ORkLrjgAnbv3k16ejruzq9//WuSkpLo168f2dnZfPbZZwA0a9aMnj17FruPv/3tbyxbtowePXowY8YMfvrTnwLQvXt39u3bx5YtW3jttdfo2bMnp5122lH1zZ8/n//85z/07t07SkdBRAQI/0e9xNcVpNaMO9218i7yPNSTyfVc7nrjLh66/KGo77devXoAxMbGcvjwYSB0nuzBBx/ksssuK7TsnDlzyMnJYf369dStW5fmzZtHfl/olFNOKXU/iYmJJCYmcu2119KiRQvmzJkDwMiRI5k/fz6bNm0qNAyZb+TIkXTp0oXrr78+ct5ORKTC3Xkn7NkDf/kLmIVCbepUaNw49F4FqhX/kuX31r7N/RaAb3O/rdReW1GXXXYZDz/8MN999x0AW7du5ZtvvuGrr77irLPOom7duixfvpyPPir2jgyF7Nu3jxUrVkRep6amFhqqHDVqFE8//TT//ve/GTJkyFHrn3/++dx9991MmjTpxD+YiEhx3EOhNnNmKMzyQ23mzFB7BffcakWPrWBvLV9F9Nr2799PXFxc5PWtt95apvXGjh1LZmYmXbp0wd1p0qQJL730EqNHj+aKK66gW7duJCcn07Zt22Nuy9257777uPHGGzn55JM55ZRTIr01gPbt29OgQQO6du1aYq/vxhtvLFPdIiLHxSzUU4NQmM2cGXo+ZcqRHlxF7s6jNMZZkbp16+ZFbzS6adMm2rVrV6b14/4cR/be7KPaz214Llm3ZlVIjSK1SXn+/olEuEPBUx55eScUama23t27FW2vFT02hZeISBXLH34saOrUqPTYasU5NhERqUIFz6lNmRLqqU2ZUvicWwWqFT02ERGpQmahqx8LnlPLP+fWuHGF99gUbCIiEn133hnqmeWHWH64ReFuERqKFBGRylE0xKJ0CyQFm4iIBIqC7QTotjXFmzNnDk2aNCE5OZn27dvz6KOPlrr8xRdfTNFf56gsF1544TGXKfjnvHXrVgYNGkSrVq1o164dw4cPj0x7VlFeeuklPvzww8jrd955hwsuuIDk5GTatWvHneFZGgpOuF0Wf/jDH465zJgxY3juuefKXbNIdaJgq2ATJkzguuuui9r23T0yyTEcuW3NP/7xD9LS0njvvfe4+OKL6dOnT+S2NflKum0NUObb1mRmZnLxxRcfs84RI0aQmprKihUr+PWvf13h//hXlNWrV5d52YMHD3L55ZczceJEtm3bxqZNm5g4cSI5OTkVWlPRYLv++uuZPXs2qampbNiwgeHDhx/XdssSbCJBUGuCbd48aN489LuBzZuHXkeDbltT2FlnnUV8fDwfffQRy5Yto3PnziQmJvLTn/6UQ4cOFVr2scceY2qB33N59NFHufXWWyOff9y4cXTo0IEBAwZw4MABIDSFWM+ePUlKSmLo0KF8+eWXQOjYT506lT59+tCuXTvWrl3Lj370I1q3bs3//M//RPaR3xsr6dgX9Pe//51evXpxxRVXRNouueQSOnbsyMGDB7nhhhtITEykc+fOLF++HAj1Xm+66abI8oMHD45Mgfa9732P2267jU6dOtGzZ08+++wzVq9ezSuvvMK0adNITk4mIyOD//73v5H/jMTGxtK+ffujanv00UcZOHAgBw4c4Omnn6ZHjx4kJydz4403kpuby/Tp0zlw4ADJycmMHj0agKeeeoqkpCQ6derEtddeG9nWypUrufDCC2nZsqV6b1Iz5d+4sjo/unbt6kV9+OGHR7WV5Omn3Rs0cA9dkhN6NGgQaj8Rp5xyylFtv/nNb/xPf/qTu7unpKT4rbfe6u7uixcv9r59+7q7+yOPPOJ33XWXu7sfPHjQu3bt6tu3b/fvvvvOv/rqK3d3z8nJ8fj4eM/Ly/MdO3a4mfnbb79dbB0/+9nPvEmTJj5y5Eh/+umnPTc3193d16xZ48nJyZH9NGnSxL/44gt3d7/++ut94cKFfsEFF/gXX3zhY8eO9RUrVnizZs08JyenxM+8Y8cOT0lJKfW4PPHEEz558mR3d8/IyPAmTZp4dna2x8XF+ZYtW9zd/dprr/W//OUvkeO0du1a37dvn7ds2dK//fZbd3fv1auXv//++75jxw6PjY319957z93dr7nmGp87d667uycmJvqKFSvc3f3222/3KVOmRLb5y1/+0t3d77//fm/atKnv3LnTDx486Oeee65//vnn7n7kz7CkY19wmalTp/r9999f7GeeMWOGjxkzxt3dN23a5Oedd54fOHCg0LFwd7/88st9+fLl7u4O+CuvvOLu7tOmTYt8J/L/bPL99re/9caNG/tVV13ls2bN8gMHDrj7ke/agw8+6FdccYUfPHjQP/zwQx88eHDkGE6cONGffPLJQp/D3X3Dhg2ekJAQ+bPevXt3ZN/Dhg3z3Nxc37hxo8fHxxf7ecvz908kWoB1Xkxm1Ioe2223wf79hdv27w+1R1tQblszdOhQkpOTGTRoEOvWrSM5OZnk5GSeeOKJYpdfsGABycnJjBo1ikceeYScnBxatGhBQkICEBpeW7lyZaF1TjnlFC699FIWLVrE5s2b+e6770hMTASgRYsWJCcnFzqWX331FXv27CElJaXYbeZP+pyYmEiHDh1o2rQp9erVo2XLloV6vUCpx74s3nzzzUivp23btjRr1oytW7eWus5JJ53E4MGDC32m4txxxx2sW7eOAQMG8Pe//50f/vCHkffmzp3La6+9xvPPP0+9evVYtmwZ69evp3v37iQnJ7Ns2TK2b99+1Db//e9/M2zYMM4880wgdF+/fFdddRUxMTG0b9++2g4hi5SmVvwe28cfl6+9IgXltjUvvvgiEBoWHTNmTKE7ChRnxIgR/PWvf428Tk1NLXX5fGPHjuUPf/gDbdu25YYbboi05x9HCB3L/KHI0uSvExMTU2j9mJiYyJ9Fvnnz5pV47PN16NCBN954o9h9eQkzJ9SpU6fQOdGC26xbty4Wvty54PejOPHx8UycOJFx48bRpEkTdu/eDUDHjh1JTU0lKyuLFi1a4O5cf/313HPPPSVuK79eK+FS64LHqqTPJVKd1Yoe2/nnl6892mrjbWvatm1LZmYm27ZtA0I9jfyeVkEXXHABn3zyCX//+9+LDeKCTj31VE477bTIucuStlkWZTn2P/7xj1m9ejWLFy+OtC1ZsoQPPviAPn36MC984nbr1q18/PHHtGnThubNm5OamkpeXh6ffPIJa9asOWYtDRs2LHS16uLFiyMBk56eTmxsLI0bNwagc+fOPPLIIwwZMoSdO3fSt29fnnvuOf773/8C8MUXX0Q+S926dSPfub59+/Lss89GAvKLL74o7yETqbZqRY/t7rth/PjCw5ENGoTaT4RuW1N29evX54knnuCaa67h8OHDdO/enQkTJhS77PDhw0lNTT1q2LQ4Tz75JBMmTGD//v20bNmyxKHRYynLsT/55JNZtGgRt9xyC7fccgt169YlKSmJmTNnMmnSJCZMmEBiYiJ16tRhzpw51KtXj4suuogWLVqQmJhIx44d6dKlyzFrGTlyJOPGjeOBBx7gueeeY+7cuUydOpUGDRpQp04d5s2bR2xsbGT5H/zgB8yYMYPLL7+c119/nd///vcMGDCAvLw86taty0MPPUSzZs0YP348SUlJdOnShXnz5nHbbbeRkpJCbGwsnTt3LvS9EanJasVtayB0FeRtt4WGH88/PxRq4YvDpJoZPHgwU6dOpW/fvlVdipRAt62R6qCk29bUiqFICIVYZmZoUunMTIVadbRnzx4SEhI4+eSTFWoictxqxVCk1AyNGzc+5pWEIiLHUmt6bCIiUjso2EREJFAUbCIiEigKNhERCRQF2wkoeDuTV199ldatW/NxFKczKTqhbr7PPvuMwYMHR25bM2jQICA0DdWWLVsKLXvLLbdw3333sWLFCsyMxx57LPLee++9h5mV61YoIiLVTe0JtqK/r1eBv7+3bNkyfv7zn7NkyRLOL+N0Jrm5uRW2/zvuuIP+/fuTlpbGhx9+yB//+EfgyHRa+fLy8njuuecYMWIEEJqGq+js/506daqwukREqkLtCLY774SpU4+EmXvodfiGjSdi1apVjBs3jsWLFxMfHw9Q7G1DINTDu+OOO7jgggt4++23i71tCUBOTg5XX3013bt3p3v37rz11lul1rBr165CM6AkJSUBoem0CgbbypUrad68eWTKrfPPP5+DBw/y2Wef4e4sWbKEgQMHnvAxERGpSsEPNnfYswdmzjwSblOnhl7v2XNCPbdDhw5x5ZVX8tJLL0WmYNq0aRMLFizgrbfeIjU1ldjY2Mgcgt988w0dO3bkP//5Dz/4wQ/45ptv6NmzJ2lpafTp0ydyp+kpU6YwdepU1q5dy/PPP8/YsWNLrWPy5Mn87Gc/45JLLuHuu+9m586dQCjgYmJiSEtLA46+HxvAsGHDWLhwIatXr6ZLly6FJsAVEamJgv8L2mbwl7+Ens+cGXoATJkSai9hhvOyqFu3LhdeeCGPPfYYM8PbLXjbEIADBw5w1llnAaEZ3K+++urI+kVvW/L6668DoTtdF7yD8tdff11oUtyiLrvsMrZv386SJUt47bXX6Ny5Mxs2bKBJkyaRXluHDh14+eWX+d3vfldo3eHDhzNixAg2b97MqFGjynVHaRGR6ij4PTYoHG75TjDUIHT7k2effZa1a9fyhz/8ASBy25DU1FRSU1PZsmULd4aHPOvXr19o8tqSbluSl5fH22+/HdlGdnY2DRs2LLWW008/nR//+MfMnTuX7t27R+5LNmrUKJ599lmWLl1KUlJSJGTznXPOOdStW5fXX39d01iJSCDUjmDLH34sqOA5txPQoEEDFi1axLx583jsscdKvW1IWQ0YMKBc9zL797//zf7wrQv27t1LRkZG5CKW+Ph4zjjjDKZPn17ibWB+97vfce+99xYKXRGRmir4wVbwnNqUKaFZkKdMKXzO7QSdfvrpLFmyhN///vekp6dHbhuSlJRE//792bVrV7m298ADD7Bu3TqSkpJo3749s2bNKnX59evX061bN5KSkujVqxdjx46NDIVCqNe2efNmhg4dWuz6F154IVdddVW5ahQRqa5qx21r7rwzdKFI/vBjftg1blwhV0aK1Da6bY1UByXdtib4F49AKLzcj5xTyz/ndoLn2EREpPoJ/lBkvqIhplATEQmkGh1sNWEYVSRo9PdOqrsaG2z169dn9+7d+ksmUoncnd27d1O/fv2qLkWkRDX2HFtcXBxZWVnk5ORUdSkitUr9+vULTeEmUt3U2GCrW7cuLVq0qOoyRESkmqmxQ5EiIiLFUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBErVgM7M2ZpZa4PG1md1iZp3M7G0z+8DM/mFmjaJVg4iI1D5RCzZ33+Luye6eDHQF9gMvAn8Dprt7Yvj1tGjVICIitU9lDUX2BTLc/SOgDbAy3P46cHUl1SAiIrVAZQXbSOCZ8PMNwJDw82uA84pbwczGm9k6M1uXk5NTCSWKiEgQRD3YzOwkQkG2MNz0U2Cyma0HGgLfFreeu892927u3q1JkybRLlNERAKiTiXsYyDwrrt/BuDum4EBAGaWAFxeCTWIiEgtURlDkaM4MgyJmZ0V/hkD/A8wqxJqEBGRWiKqwWZmDYD+wAsFmkeZ2VZgM7ATeCKaNYiISO0S1aFId98PnFGkbSYwM5r7FRGR2kszj4iISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2ERE5pmnTptG2bVuSkpIYOnQoe/bsibx3zz330KpVK9q0acM///nPSPv69etJTEykVatW3Hzzzbg7AIcOHWLEiBG0atWKCy64gMzMzAqtVcEmIiLH1L9/fzZs2MD7779PQkIC99xzDwAffvgh8+fPZ+PGjSxZsoRJkyaRm5sLwMSJE5k9ezbp6emkp6ezZMkSAB577DFOO+00tm3bxtSpU/nVr35VobUq2EREoigzM5N27doxbtw4OnTowIABAzhw4ECxy27bto1+/frRqVMnunTpQkZGBu7OtGnT6NixI4mJiSxYsACAFStWkJKSwvDhw0lISGD69OnMmzePHj16kJiYSEZGBgBjxoxhwoQJ9O7dm4SEBBYtWnRcn2PAgAHUqVMHgJ49e5KVlQXAyy+/zMiRI6lXrx4tWrSgVatWrFmzhl27dvH111/Tq1cvzIzrrruOl156KbLO9ddfD8CwYcNYtmxZpDdXERRsIiJRlp6ezuTJk9m4cSONGzfm+eefL3a50aNHM3nyZNLS0li9ejVNmzblhRdeIDU1lbS0NJYuXcq0adPYtWsXAGlpacycOZMPPviAuXPnsnXrVtasWcPYsWN58MEHI9vNzMzkjTfeYPHixUyYMIGDBw8ete/evXuTnJx81GPp0qVHLfv4448zcOBAALKzsznvvPMi78XFxZGdnU12djZxcXFHtRddp06dOpx66qns3r27vIe1RHUqbEsiIlKsFi1akJycDEDXrl2LPae0d+9esrOzGTp0KAD169cH4M0332TUqFHExsZy9tlnk5KSwtq1a2nUqBHdu3enadOmAMTHxzNgwAAAEhMTWb58eWTbw4cPJyYmhtatW9OyZUs2b94cqSffqlWryvRZ7r77burUqcPo0aMBiu1pmVmJ7aWtU1EUbCIiUVavXr3I89jY2GKHIksaiittiK7gdmNiYiKvY2JiOHz4cOS9oqFRXIj07t2bvXv3HtU+Y8YM+vXrB8CTTz7JokWLWLZsWWQbcXFxfPLJJ5Hls7Ky+P73v09cXFxkuLJge8F14uLiOHz4MF999RWnn356iZ+zvDQUKSJSDTRq1Ii4uLjIeahDhw6xf/9++vTpw4IFC8jNzSUnJ4eVK1fSo0ePcm174cKF5OXlkZGRwfbt22nTps1Ry6xatYrU1NSjHvmhtmTJEu69915eeeUVGjRoEFlvyJAhzJ8/n0OHDrFjxw7S09Pp0aMHTZs2pWHDhrzzzju4O0899RRXXnllZJ0nn3wSgOeee45LL720QntsCjYRkWpi7ty5PPDAAyQlJXHhhRfy6aefMnToUJKSkujUqROXXnop9913H+ecc065ttumTRtSUlIYOHAgs2bNon79+uzcuZNBgwaVeRs33XQTe/fupX///iQnJzNhwgQAOnTowPDhw2nfvj0//OEPeeihh4iNjQXg4YcfZuzYsbRq1Yr4+PjIebmf/exn7N69m1atWvHnP/+ZP/7xj+X6PMdiFXklSrR069bN161bV9VliIjUOGPGjGHw4MEMGzasqkupcGa23t27FW1Xj01ERAJFF4+IiFSyyZMn89ZbbxVqmzJlCjfccEOF72vOnDkVvs3qTsEmIlLJHnrooaouIdA0FCkiIoGiYBOpRAsXLqRDhw7ExMRQ9IKo6jaRrNQ88+ZB8+YQExP6OW9eVVdUNUoNNjOLM7P/z8xeNrO1ZrbSzP7PzC43M4WiSDl17NiRF154gT59+hRqr44TyUrNMm8ejB8PH30E7qGf48fXznArMZzM7AngceBb4F5gFDAJWAr8EHjTzPqUtL5IRQrKRLLt2rUr9pdjq+NEslKz3HYb7N9fuG3//lB7bVPaxSP/6+4bimnfALxgZicB55e0spm1ARYUaGoJ3AGsAGYB9YHDwCR3X1POuqUWSk9P55lnnuHRRx9l+PDhPP/88/zkJz85arnRo0czffp0hg4dysGDB8nLyys0keznn39O9+7dI72mtLQ0Nm3axOmnn07Lli0ZO3Ysa9asYebMmTz44IPcf//9wJGJZDMyMrjkkkvYtm1bZD6/fGWZlqg42dnZ9OzZM/I6f8LYunXrlnsi2TPPPLOMR1SC5OOPy9ceZCUGW3GhZmbxQAN3/8DdvwW2lbL+FiA5vF4skA28CDwK/NbdXzOzQcB9wMUn8BmklgjSRLJFVceJZKVmOf/80PBjce21TZkv9zezXwOJQJ6Z5bn7teXYT18gw90/MjMHGoXbTwV2lmM7UosFZSLZ4lTHiWSlZrn77tA5tYLDkQ0ahNprm9LOsf083NPK18ndR7n7aKBTOfczEngm/PwW4E9m9gkwA/j/S9j/eDNbZ2brcnJyyrk7qa2q+0SyJamOE8lKzTJ6NMyeDc2agVno5+zZofbaprQrG78ElpjZFeHX/zKzN8xsFfDPUtYrJHwubgiwMNw0EZjq7ucBU4HHilvP3We7ezd379akSZOy7k6kWk8k++KLLxIXF8fbb7/N5ZdfzmWXXQZUz4lkpeYZPRoyMyEvL/SzNoYaHGMSZDOrD0wDuhG68CMdqOvuX5V5B2ZXApPdfUD49VdAY3d3C/338it3b1TaNjQJslS1IE8kK1JTHe8kyPGErmy8EbgJuB84uZz7HsWRYUgInVNLCT+/lFBYioiIVIgSLx4xsznh908mdOHHODPrDDxqZmvc/a5jbdzMGgD9CQVjvnHATDOrAxwExp9A/VKLaSJZESlOiUORZpbm7p3Cz99z984F3rvS3V+upBo1FCkiIkcpaSiytMv9l5jZG8BJwN8LvlGZoSYiIlIeJZ5jc/dfAVcA/d39T5VXkkjV0kSyIjVbab/H9hNgn7vvK+H9eDP7QdQqE6kCmkhWpOYrbSjyDOA9M1sPrAdyCM3v2IrQVY2fA9OjXqFIJSptItna+jtBIjVNaXNFzjSzvxK6JP8iIAk4AGwCrnX3Wji1pgSdJpIVqflKnSvS3XOB18MPkcDTRLIiNZ9uFipSwN13hyaOLai2TiQrUlMp2EQK0ESyIjXfMW9bY2ax4SFJkVph9GgFmUhNVpYe2zYz+5OZtY96NSIiIieoLMGWBGwF/mZm74Tvk1bqbPwiIiJV5ZjB5u573f1Rd78Q+CXwG2CXmT1pZq2iXqGIiEg5HDPYzCzWzIaY2YvATOB/gZbAP4BXo1yfiIhIuRzz4hFC90tbDvzJ3VcXaH/OzPpEpywREZHjU2qwmVksMMfdf1fc++5+c1SqEhEROU6lDkWGL/O/pJJqEREROWFlGYpcHZ4zcgHwTX6ju78btapERESOU1mC7cLwz4LDkU5ocmQREZFq5ZjB5u4aihQRkRqjLD02zOxyoAOh+7EBUNIFJSIiIlWpLL/HNgsYAfwcMOAaoFmU6xIRETkuZZlS60J3vw740t1/C/QCzotuWSIiIsenLMF2MPxzv5l9H/gOaBG9kkRERI5fWc6x/cPMGgN/At4ldEXko9EsSkRE5HiV2mMzsxhgmbvvcffnCZ1ba+vud1RKdVKr3HnnnZx77rkkJyeTnJzMq68emYr0nnvuoVWrVrRp04Z//vOfkfb169eTmJhIq1atuPnmm3H3qihdRKqRY808kkdo0uP814fc/auoVyW11tSpU0lNTSU1NZVBgwYB8OGHHzJ//nw2btzIkiVLmDRpErm5oXvfTpw4kdmzZ5Oenk56ejpLliypyvJFpBooyzm2f5nZ1WZmUa9GjktmZibt2rVj3LhxdOjQgQEDBnDgwIFil922bRv9+vWjU6dOdOnShYyMDNydadOm0bFjRxITE1mwYAEAK1asICUlheHDh5OQkMD06dOZN28ePXr0IDExkYyMDADGjBnDhAkT6N27NwkJCSxatKhCP9/LL7/MyJEjqVevHi1atKBVq1asWbOGXbt28fXXX9OrVy/MjOuuu46XXnqpQvctIjVPWYLtVmAhcMjMvjazvWb2dZTrknJKT09n8uTJbNy4kcaNG/P8888Xu9zo0aOZPHkyaWlprF69mqZNm/LCCy+QmppKWloaS5cuZdq0aezatQuAtLQ0Zs6cyQcffMDcuXPZunUra9asYezYsTz44IOR7WZmZvLGG2+wePFiJkyYwMGDB4/ad+/evSPDjAUfS5cujSzz17/+laSkJH7605/y5ZdfApCdnc155x25EDcuLo7s7Gyys7OJi4s7ql1EareyzDzSsDIKkRPTokULkpOTAejatSuZmZlHLbN3716ys7MZOnQoAPXrh37f/s0332TUqFHExsZy9tlnk5KSwtq1a2nUqBHdu3enadOmAMTHxzNgwAAAEhMTWb58eWTbw4cPJyYmhtatW9OyZUs2b94cqSffqlWrSv0MEydO5Pbbb8fMuP322/nFL37B448/Xux5MzMrsV1EardjBltJ91xz95UVX44cr3r16kWex8bGFjsUWdKFFaVdcFFwuzExMZHXMTExHD58OPJe0UApLmB69+7N3r17j2qfMWMG/fr14+yzz460jRs3jsGDBwOhntgnn3wSeS8rK4vvf//7xMXFkZWVdVS7iNRuZRmKnFbgcTuhO2ffGcWaJEoaNWpEXFxc5DzUoUOH2L9/P3369GHBggXk5uaSk5PDypUr6dGjR7m2vXDhQvLy8sjIyGD79u20adPmqGVWrVoVuTCk4KNfv34AkeFPgBdffJGOHTsCMGTIEObPn8+hQ4fYsWMH6enp9OjRg6ZNm9KwYUPeeecd3J2nnnqKK6+88jiPjogERVmGIq8o+NrMzgPui1pFElVz587lxhtv5I477qBu3bosXLiQoUOH8vbbb9OpUyfMjPvuu49zzjmHzZs3l3m7bdq0ISUlhc8++4xZs2ZRv359du7cydixYwtdtl+aX/7yl6SmpmJmNG/enEceeQSADh06MHz4cNq3b0+dOnV46KGHiI2NBeDhhx9mzJgxHDhwgIEDBzJw4MDyHxQRCRQr7+/9hK+OfN/dE6NT0tG6devm69atq6zdSTmNGTOGwYMHM2zYsKouRURqETNb7+7diraX5Rzbg4RmG4HQ0GUykFah1YmIiFSQskypVbCrdBh4xt3filI9UkEmT57MW28V/mOaMmUKN9xwQ4Xva86cORW+TRGR41WWYHsOOOjuuQBmFmtmDdx9f3RLkxPx0EMPVXUJIiJVoixXRS4DTi7w+mRgaQnLioiIVKmyBFt9d9+X/yL8vEH0SpLqbN48aN4cYmJCP+fNq+qKREQKK0uwfWNmXfJfmFlXoPiJCCXQ5s2D8ePho4/APfRz/HiFm4hUL8e83N/MugPzgZ3hpqbACHdfH+XaInS5f/XQvHkozIpq1gyKmcFLRCSqjvtyf3dfa2ZtgTaAAZvd/bso1CjV3Mcfl69dRKQqHHMo0swmA6e4+wZ3/wD4nplNin5pUt2cf3752kVEqkJZzrGNc/c9+S/c/UtgXNQqkmrr7ruhQZHLhho0CLWLiFQXZQm2mII3GTWzWOCk6JUk1dXo0TB7duicmlno5+zZoXYRkeqiLL+g/U/gWTObRWhqrQnAkqhWJdXW6NEKMhGp3soSbL8CxgMTCV088i/g0WgWJSIicryOORTp7nnuPsvdh7n71cBG4MHolyYiIlJ+ZemxYWbJwChgBLADeCGKNYmIiBy3EoPNzBKAkYQCbTewgNAvdF9SSbWJiIiUW2k9ts3AKuAKd98GYGZTK6UqERGR41TaObargU+B5Wb2qJn1JXTxiIiISLVVYrC5+4vuPgJoC6wApgJnm9nDZjagkuoTEREpl7JcFfmNu89z98FAHJAKTI92YSIiIsejLDOPRLj7F+7+iLtfGq2CRERETkS5gk1ERKS6U7CJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKHWitWEzawMsKNDUErgD6AW0Cbc1Bva4e3K06hARkdolasHm7luAZAAziwWygRfd/f78Zczsf4GvolWDiIjUPlELtiL6Ahnu/lF+g5kZMBy4tJJqEBGRWqCyzrGNBJ4p0tYb+Mzd04tbwczGm9k6M1uXk5MT9QJFRCQYoh5sZnYSMARYWOStURwddhHuPtvdu7l7tyZNmkSzRBERCZDKGIocCLzr7p/lN5hZHeBHQNdK2L+IiNQilTEUWVzPrB+w2d2zKmH/IiJSi0Q12MysAdAfeKHIW8WdcxMRETlhUR2KdPf9wBnFtI+J5n5FRKT20swjIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcFWC02bNo22bduSlJTE0KFD2bNnDwCZmZmcfPLJJCcnk5yczIQJEyLrrF+/nsTERFq1asXNN9+Mu1dR9SIipVOw1UL9+/dnw4YNvP/++yQkJHDPPfdE3ouPjyc1NZXU1FRmzZoVaZ84cSKzZ88mPT2d9PR0lixZUhWli4gck4KtHDIzM2nXrh3jxo2jQ4cODBgwgAMHDhS77LZt2+jXrx+dOnWiS5cuZGRk4O5MmzaNjh07kpiYyIIFCwBYsWIFKSkpDB8+nISEBKZPn868efPo0aMHiYmJZGRkADBmzBgmTJhA7969SUhIYNGiRcf1OQYMGECdOnUA6NmzJ1lZWaUuv2vXLr7++mt69eqFmXHdddfx0ksvHde+RUSiTcFWTunp6UyePJmNGzfSuHFjnn/++WKXGz16NJMnTyYtLY3Vq1fTtGlTXnjhBVJTU0lLS2Pp0qVMmzaNXbt2AZCWlsbMmTP54IMPmDt3Llu3bmXNmjWMHTuWBx98MLLdzMxM3njjDRYvXsyECRM4ePDgUfvu3bt3ZDix4GPp0qVHLfv4448zcODAyOsdO3bQuXNnUlJSWLVqFQDZ2dnExcVFlomLiyM7O/v4DqCISJTVqeoCapoWLVqQnJwMQNeuXcnMzDxqmb1795Kdnc3QoUMBqF+/PgBvvvkmo0aNIjY2lrPPPpuUlBTWrl1Lo0aN6N69O02bNgVCw4EDBgwAIDExkeXLl0e2PXz4cGJiYmjdujUtW7Zk8+bNkXry5QfSsdx9993UqVOH0aNHA9C0aVM+/vhjzjjjDNavX89VV13Fxo0biz2fZmZl2oeISGVTsJVTvXr1Is9jY2OLHYos6cKK0i64KLjdmJiYyOuYmBgOHz4cea9ooBQXML1792bv3r1Htc+YMYN+/foB8OSTT7Jo0SKWLVsW2Ua9evUi++3atSvx8fFs3bqVuLi4QsOVWVlZfP/73y/xs4iIVCUNRUZBo0aNiIuLi5yHOnToEPv376dPnz4sWLCA3NxccnJyWLlyJT169CjXthcuXEheXh4ZGRls376dNm3aHLXMqlWrIheAFHzkh9qSJUu49957eeWVV2jQoEFkvZycHHJzcwHYvn076enptGzZkqZNm9KwYUPeeecd3J2nnnqKK6+88jiPjohIdCnYomTu3Lk88MADJCUlceGFF/Lpp58ydOhQkpKS6NSpE5deein33Xcf55xzTrm226ZNG1JSUhg4cCCzZs2ifv367Ny5k0GDBpV5GzfddBN79+6lf//+hS7rX7lyZaS+YcOGMWvWLE4//XQAHn74YcaOHUurVq2Ij48vdF5ORKQ6sZrw+0jdunXzdevWVXUZVW7MmDEMHjyYYcOGVXUpIiJVzszWu3u3ou3qsYmISKDo4pETNHnyZN56661CbVOmTOGGG26o8H3NmTOnwrcpIhI0CrYT9NBDD1V1CSIiUoCGIkVEJFAUbNXUpElQpw6YhX5OmlTVFYmI1AwaiqyGJk2Chx8+8jo398jr//u/qqlJRKSmUI+tGpo9u3ztIiJyhIKtGgpP/lHmdhEROULBVg3FxpavXUREjlCwVUPjx5evXUREjtDFI9VQ/gUis2eHhh9jY0OhpgtHRESOTcFWTf3f/ynIRESOh4YiRUQkUKIWbGbWxsxSCzy+NrNbwu/93My2mNlGM7svWjWIiEjtE7WhSHffAiQDmFkskA28aGaXAFcCSe5+yMzOilYNIiJS+1TWUGRfIMPdPwImAn9090MA7v7fSqpBRERqgcoKtpHAM+HnCUBvM/uPmb1hZt2LW8HMxpvZOjNbl5OTU0lliohITRf1YDOzk4AhwMJwUx3gNKAnMA141sys6HruPtvdu7l7tyZNmkS7TBERCYjK6LENBN5198/Cr7OAFzxkDZAHnFkJdYiISC1QGcE2iiPDkAAvAZcCmFkCcBLweSXUISIitUBUg83MGgD9gRcKND8OtDSzDcB84Hp392jWISIitUdUZx5x9/3AGUXavgV+Es39iohI7aWZR0REJFAUbCIiEigKNhERCRQFm4iIBIqCTUREAkXBJiIigaJgExGRQFGwiYhIoCjYREQkUBRsIiISKAo2EREJFAWbiIgEioJNREQCRcEmIiKBomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUCpNcG2a+8u4h+I59N9n1Z1KSIiEkW1JtjuWnkXmXsyueuNu6q6FBERiaJaEWy79u7iidQnyPM8nkh9Qr02EZEAqxXBdtfKu8jzPAByPVe9NhGRAAt8sOX31r7N/RaAb3O/Va9NRCTAAh9sBXtr+dRrExEJrsAH2ytbXon01vJ9m/stL295uYoqEhGRaKpT1QVEW9atWVVdgoiIVKLA99hERKR2UbCJiEigKNhERCRQFGwiIhIoCjYREQkUBZuIiASKgk1ERAJFwSYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEijm7lVdwzGZWQ7w0XGseibweQWXE3Q6ZsdHx638dMyOj47bEc3cvUnRxhoRbMfLzNa5e7eqrqMm0TE7Pjpu5adjdnx03I5NQ5EiIhIoCjYREQmUoAfb7KouoAbSMTs+Om7lp2N2fHTcjiHQ59hERKT2CXqPTUREahkFm4iIBEqNDzYz+6GZbTGzbWY2vZj3zcweCL//vpl1qYo6q5syHLeLzewrM0sNP+6oijqrEzN73Mz+a2YbSnhf37UiynDM9D0rhpmdZ2bLzWyTmW00synFLKPvW0ncvcY+gFggA2gJnASkAe2LLDMIeA0woCfwn6quu6ofZTxuFwOLqrrW6vQA+gBdgA0lvK/vWvmPmb5nxR+XpkCX8POGwFb921b2R03vsfUAtrn7dnf/FpgPXFlkmSuBpzzkHaCxmTWt7EKrmbIcNynC3VcCX5SyiL5rRZThmEkx3H2Xu78bfr4X2AScW2Qxfd9KUNOD7VzgkwKvszj6D78sy9Q2ZT0mvcwszcxeM7MOlVNajabv2vHR96wUZtYc6Az8p8hb+r6VoE5VF3CCrJi2or+/UJZlapuyHJN3Cc3Dts/MBgEvAa2jXVgNp+9a+el7Vgoz+x7wPHCLu39d9O1iVtH3jZrfY8sCzivwOg7YeRzL1DbHPCbu/rW77ws/fxWoa2ZnVl6JNZK+a+Wk71nJzKwuoVCb5+4vFLOIvm8lqOnBthZobWYtzOwkYCTwSpFlXgGuC19B1BP4yt13VXah1cwxj5uZnWNmFn7eg9B3ZXelV1qz6LtWTvqeFS98TB4DNrn7n0tYTN+3EtTooUh3P2xmNwH/JHSl3+PuvtHMJoTfnwW8SujqoW3AfuCGqqq3uijjcRsGTDSzw8ABYKSHL8WqrczsGUJX8Z1pZlnAb4C6oO9aScpwzPQ9K95FwLXAB2aWGm77NXA+6Pt2LJpSS0REAqWmD0WKiIgUomATEZFAUbCJiEigKNhERCRQFGwiIhIoCjaRMDM7o8As85+aWXaB1ycdY91uZvZAGfaxuoJqTQ7P1JH/ekhxd2mooH3db2Z9Snn/JjPTpeZSbehyf5FimNmdwD53n1GgrY67H666qo4wszFAN3e/Kcr7OR141d17lrJMA+Atd+8czVpEyko9NpFSmNkcM/uzmS0H7jWzHma22szeC/9sE17uYjNbFH5+Z/g+ZCvMbLuZ3Vxge/sKLL/CzJ4zs81mNq/ADByDwm1vhu+3tahITScBvwNGhHuTI8xsjJn9tUDND1vofl7bzSwlXM8mM5tTYDsDzOxtM3vXzBaG5yUsahiwpMA6fzSzDy10/68ZAO6+H8gMzxwiUuVq9MwjIpUkAejn7rlm1gjoE569pR/wB+DqYtZpC1xC6F5aW8zsYXf/rsgynYEOhOb3ewu4yMzWAY+E97EjPHNHIe7+rYVuyBnpsYV7cAWdBlwKDAH+QWgmi7HAWjNLJjTP4P+EP9c3ZvYr4FZCgVnQRcBz4X2cDgwF2rq7m1njAsutA3oDa4o5FiKVSsEmcmwL3T03/PxU4Ekza01oJvW6Jayz2N0PAYfM7L/A2YTCpKA17p4FEJ42qTmwD9ju7jvCyzwDjD+Omv8RDp8PgM/c/YPwfjaG9xMHtAfeCncUTwLeLmY7TYGc8POvgYPA38xsMVCwJ/lfQmEuUuUUbCLH9k2B53cBy919qIXuk7WihHUOFXieS/F/14pbprhbkRyP/G3nFdlPXng/ucDr7j7qGNs5ANSHyByjPYC+hCbOvolQr5DwMgcqpnSRE6NzbCLlcyqQHX4+Jgrb3wy0DIcmwIgSlttLaJjzeL1DaOizFYQuADGzhGKW2wTkL/M94NTw7WVuAZILLJcAbDiBekQqjIJNpHzuA+4xs7cI3RmhQrn7AWASsMTM3gQ+A74qZtHlQPv8i0eOYz85hIL5GTN7n1DQFTeUuJjQ7PwQCtJF4eXfAKYWWO4iYGl56xCJBl3uL1LNmNn3wneUNuAhIN3d/1KF9bwJDHb3PSW83xm41d2vrdTCREqgHptI9TMufDHJRkJDn49UbTn8gvB9wEpwJnB7JdUickzqsYmISKCoxyYiIoGiYBMRkUBRsImISKAo2EREJFAUbCIiEij/D42R6mOYUiC0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N_COMPONENTS = [250, 500, 1000, 2000]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 7))\n",
    "ax.scatter([results[\"LSVM\"][\"time\"], ], [results[\"LSVM\"][\"score\"], ],\n",
    "           label=\"Linear SVM\", c=\"green\", marker=\"^\")\n",
    "\n",
    "ax.scatter([results[\"LSVM + PS(250)\"][\"time\"], ],\n",
    "           [results[\"LSVM + PS(250)\"][\"score\"], ],\n",
    "           label=\"Linear SVM + PolynomialCountSketch\", c=\"blue\")\n",
    "for n_components in N_COMPONENTS:\n",
    "    ax.scatter([results[f\"LSVM + PS({n_components})\"][\"time\"], ],\n",
    "               [results[f\"LSVM + PS({n_components})\"][\"score\"], ],\n",
    "               c=\"blue\")\n",
    "    ax.annotate(f\"n_comp.={n_components}\",\n",
    "                (results[f\"LSVM + PS({n_components})\"][\"time\"],\n",
    "                 results[f\"LSVM + PS({n_components})\"][\"score\"]),\n",
    "                xytext=(-30, 10), textcoords=\"offset pixels\")\n",
    "\n",
    "ax.scatter([results[\"KSVM\"][\"time\"], ], [results[\"KSVM\"][\"score\"], ],\n",
    "           label=\"Kernel SVM\", c=\"red\", marker=\"x\")\n",
    "\n",
    "ax.set_xlabel(\"Training time (s)\")\n",
    "ax.set_ylabel(\"Accurary (%)\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "[1] Pham, Ninh and Rasmus Pagh. \"Fast and scalable polynomial kernels via\n",
    "explicit feature maps.\" KDD '13 (2013).\n",
    "https://doi.org/10.1145/2487575.2487591\n",
    "\n",
    "[2] LIBSVM binary datasets repository\n",
    "https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-06-10 16:36:41,186\tINFO services.py:1267 -- View the Ray dashboard at \u001b[1m\u001b[32mhttp://127.0.0.1:8266\u001b[39m\u001b[22m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAGpCAYAAADhiRM+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA3HUlEQVR4nO3deXxU1f3/8ddJgsQoiiJqNMqSENaEYRWwEJXFgohSkaVURQsawIrYLy2/2lpbq1bLtxUtX1mqohgFUVwKSisIoqJl0URBlhCImoA0omiQRUk+vz9mMiYhCUnIZLnzfj4eeTBz5s69n7kMvj3n3pzjzAwRERGviKjrAkRERGqSgk1ERDxFwSYiIp6iYBMREU9RsImIiKdE1XUBlXHWWWdZy5Yt67oMERGpRzZu3PiFmTUv3d4ggq1ly5Zs2LChrssQEZF6xDn3SVntGooUERFPUbCJiIinKNhERMRTGsQ1trJ8//335OTkcPjw4bouRSSsREdHExcXR6NGjeq6FJEyNdhgy8nJoUmTJrRs2RLnXF2XIxIWzIx9+/aRk5NDq1at6rockTI12KHIw4cP06xZM4WaSC1yztGsWTONlEi91mCDDVCoidQB/buT+q5BB5uIiEhpCrYTcOqppx7TNnv2bJ566qlarWPp0qV06dKFzp0706FDB+bMmcPq1avp3bt3ie2OHj3KOeecw549exg3bhwxMTHk5+cHX58yZQrOOb744otarV9EpCY12JtH6qvU1NSQ7t/MMDMiIvz/T/L9999z8803s27dOuLi4jhy5AjZ2dm0adOGnJwcsrOzKZqObMWKFXTq1InY2FgAEhISePnll/nZz35GYWEhq1at4vzzzw9p/SIioRZWPbY9+XuIfziezw98HrJj3H333cyYMQOASy65hF//+tf07NmTxMRE3nrrLQAKCgqYNm0aPXr0IDk5mTlz5gBw4MAB+vfvT9euXUlKSuLll18GIDs7m/bt2zNp0iS6du3KZ599Fjxefn4+R48epVmzZgA0btyYtm3bEhERwbXXXsuiRYuC2y5cuJAxY8YEn48ZMyb4+urVq7n44ouJitL/64hIwxZWwXbPmnvI3p/NPW/eU2vHPHr0KOvWreOhhx7iD3/4AwCPPfYYp59+OuvXr2f9+vXMmzePXbt2ER0dzYsvvsj777/PqlWr+OUvf4mZAbBt2zauv/56PvjgA1q0aBHc/5lnnsmwYcNo0aIFY8aMIS0tjcLCQsAfXAsXLgTgyJEjvPrqq1xzzTXB97Zp04a8vDy++uornn32WUaPHl1bp0VEJGTCJtj25O/hifQnKLRCnkh/IqS9tuJ+8pOfANCtWzeys7MB+Pe//81TTz2Fz+fjoosuYt++fWRmZmJm/OY3vyE5OZkBAwaQm5vL3r17AWjRogW9evUq8xj/+Mc/WLlyJT179mTGjBncdNNNAPTo0YMDBw6wbds2XnvtNXr16sUZZ5xxTH0LFy7kP//5D3379g3RWRARAQL/o17u8xoSNuNO96y5h0Lz92QKrIB73ryHWVfMCvlxGzduDEBkZCRHjx4F/NfJHnnkES6//PIS286fP5+8vDw2btxIo0aNaNmyZfD3hU455ZQKj5OUlERSUhLXXXcdrVq1Yv78+QCMHj2ahQsXsmXLlhLDkEVGjx5N165dueGGG4LX7UREatzdd8P+/fC3v4Fz/lCbOhWaNvW/VoPC4r9kRb217wq+A+C7gu9qtddW2uWXX86jjz7K999/D8D27dv59ttv+frrrzn77LNp1KgRq1at4pNPylyRoYQDBw6wevXq4PP09PQSQ5Vjxozh6aef5o033mDYsGHHvP/CCy/k3nvvZdKkSSf+wUREymLmD7WZM/1hVhRqM2f622u45xYWPbbivbUiNdFrO3jwIHFxccHnd9xxR6XeN378eLKzs+natStmRvPmzXnppZcYO3YsV155Jd27d8fn89GuXbvj7svMePDBB7nllls4+eSTOeWUU4K9NYAOHToQExNDt27dyu313XLLLZWqW0SkWpzz99TAH2YzZ/ofT5nyQw+uJg9nIRrjrEndu3e30guNbtmyhfbt21fq/XF/jSM3P/eY9vObnE/OHTk1UqNIOKnKvz+RIDMofsmjsPCEQs05t9HMupduD4sem8JLRKSOFQ0/Fjd1akh6bGFxjU1EROpQ8WtqU6b4e2pTppS85laDwqLHJiIidcg5/92Pxa+pFV1za9q0xntsCjYREQm9u+/298yKQqwo3EKwWoSGIkVEpHaUDrEQLYGkYBMREU9RsJ0ALVtTtvnz59O8eXN8Ph8dOnRg3rx5FW5/ySWXUPrXOWpLnz59jrtN8b/n7du3M2TIEBISEmjfvj0jR44MTntWU1566SU+/vjj4PP33nuPiy66CJ/PR/v27bk7MEtD8Qm3K+O+++477jbjxo3j+eefr3LNIvWJgq2Gpaamcv3114ds/2YWnOQYfli25p///CcZGRl88MEHXHLJJfTr1y+4bE2R8patASq9bE12djaXXHLJcescNWoU6enprF69mt/85jc1/h//mrJ27dpKb3v48GGuuOIKJk6cyI4dO9iyZQsTJ04kLy+vRmsqHWw33HADc+fOJT09nU2bNjFy5Mhq7bcywSbiBWETbGlp0LKl/3cDW7b0Pw8FLVtT0tlnn018fDyffPIJK1eupEuXLiQlJXHTTTdx5MiREts+9thjTC32ey7z5s3jjjvuCH7+CRMm0LFjRwYNGsShQ4cA/xRivXr1Ijk5meHDh/PVV18B/nM/depU+vXrR/v27Vm/fj0/+clPaNOmDb/97W+DxyjqjZV37ot75pln6N27N1deeWWw7dJLL6VTp04cPnyYG2+8kaSkJLp06cKqVasAf+/11ltvDW4/dOjQ4BRop556KnfeeSedO3emV69e7N27l7Vr1/LKK68wbdo0fD4fWVlZ/Pe//w3+z0hkZCQdOnQ4prZ58+YxePBgDh06xNNPP03Pnj3x+XzccsstFBQUMH36dA4dOoTP52Ps2LEAPPXUUyQnJ9O5c2euu+664L7WrFlDnz59aN26tXpv0jAVLVxZn3+6detmpX388cfHtJXn6afNYmLM/Lfk+H9iYvztJ+KUU045pu33v/+9/eUvfzEzs5SUFLvjjjvMzGzZsmXWv39/MzObM2eO3XPPPWZmdvjwYevWrZvt3LnTvv/+e/v666/NzCwvL8/i4+OtsLDQdu3aZc45e/fdd8us4+c//7k1b97cRo8ebU8//bQVFBSYmdm6devM5/MFj9O8eXP78ssvzczshhtusMWLF9tFF11kX375pY0fP95Wr15tLVq0sLy8vHI/865duywlJaXC8/LEE0/Y5MmTzcwsKyvLmjdvbrm5uRYXF2fbtm0zM7PrrrvO/va3vwXP0/r16+3AgQPWunVr++6778zMrHfv3vbhhx/arl27LDIy0j744AMzM7v22mttwYIFZmaWlJRkq1evNjOz3/3udzZlypTgPn/1q1+ZmdlDDz1ksbGxtnv3bjt8+LCdf/759sUXX5jZD3+H5Z374ttMnTrVHnrooTI/84wZM2zcuHFmZrZlyxa74IIL7NChQyXOhZnZFVdcYatWrTIzM8BeeeUVMzObNm1a8DtR9HdT5A9/+IM1bdrUrr76aps9e7YdOnTIzH74rj3yyCN25ZVX2uHDh+3jjz+2oUOHBs/hxIkT7cknnyzxOczMNm3aZImJicG/63379gWPPWLECCsoKLDNmzdbfHx8mZ+3Kv/+REIF2GBlZEZY9NjuvBMOHizZdvCgvz3UvLJszfDhw/H5fAwZMoQNGzbg8/nw+Xw88cQTZW6/aNEifD4fY8aMYc6cOeTl5dGqVSsSExMB//DamjVrSrznlFNO4bLLLmPp0qVs3bqV77//nqSkJABatWqFz+crcS6//vpr9u/fT0pKSpn7LJr0OSkpiY4dOxIbG0vjxo1p3bp1iV4vUOG5r4y333472Otp164dLVq0YPv27RW+56STTmLo0KElPlNZ7rrrLjZs2MCgQYN45pln+PGPfxx8bcGCBbz22mu88MILNG7cmJUrV7Jx40Z69OiBz+dj5cqV7Ny585h9vvHGG4wYMYKzzjoL8K/rV+Tqq68mIiKCDh061NshZJGKhMXvsX36adXaa5JXlq158cUXAf+w6Lhx40qsKFCWUaNG8fe//z34PD09vcLti4wfP5777ruPdu3aceONNwbbi84j+M9l0VBkRYreExERUeL9ERERwb+LImlpaeWe+yIdO3bkzTffLPNYVs7MCVFRUSWuiRbfZ6NGjXCB252Lfz/KEh8fz8SJE5kwYQLNmzdn3759AHTq1In09HRycnJo1aoVZsYNN9zA/fffX+6+iup15dxqXfxclfe5ROqzsOixXXhh1dpDLRyXrWnXrh3Z2dns2LED8Pc0inpaxV100UV89tlnPPPMM2UGcXGnn346Z5xxRvDaZXn7rIzKnPuf/vSnrF27lmXLlgXbli9fzkcffUS/fv1IC1y43b59O59++ilt27alZcuWpKenU1hYyGeffca6deuOW0uTJk1K3K26bNmyYMBkZmYSGRlJ06ZNAejSpQtz5sxh2LBh7N69m/79+/P888/z3//+F4Avv/wy+FkaNWoU/M7179+f5557LhiQX375ZVVPmUi9FRY9tnvvhZtvLjkcGRPjbz8RWram8qKjo3niiSe49tprOXr0KD169CA1NbXMbUeOHEl6evoxw6ZlefLJJ0lNTeXgwYO0bt263KHR46nMuT/55JNZunQpt99+O7fffjuNGjUiOTmZmTNnMmnSJFJTU0lKSiIqKor58+fTuHFjLr74Ylq1akVSUhKdOnWia9eux61l9OjRTJgwgYcffpjnn3+eBQsWMHXqVGJiYoiKiiItLY3IyMjg9j/60Y+YMWMGV1xxBa+//jp/+tOfGDRoEIWFhTRq1IhZs2bRokULbr75ZpKTk+natStpaWnceeedpKSkEBkZSZcuXUp8b0QasrBYtgb8d0Heead/+PHCC/2hFrg5TOqZoUOHMnXqVPr371/XpUg5tGyN1AflLVsTFkOR4A+x7Gz/pNLZ2Qq1+mj//v0kJiZy8sknK9REpNrCYihSGoamTZse905CEZHjCZsem4iIhAcFm4iIeIqCTUREPEXBJiIinqJgOwHFlzN59dVXadOmDZ+GcDqT0hPqFtm7dy9Dhw4NLlszZMgQwD8N1bZt20pse/vtt/Pggw+yevVqnHM89thjwdc++OADnHNVWgpFRKS+CZ9gK/37ejX4+3srV67kF7/4BcuXL+fCSk5nUlBQUGPHv+uuuxg4cCAZGRl8/PHH/PnPfwZ+mE6rSGFhIc8//zyjRo0C/NNwlZ79v3PnzjVWl4hIXQiPYLv7bpg69YcwM/M/DyzYeCLeeustJkyYwLJly4iPjwcoc9kQ8Pfw7rrrLi666CLefffdMpctAcjLy+Oaa66hR48e9OjRg3feeafCGvbs2VNiBpTk5GTAP51W8WBbs2YNLVu2DE65deGFF3L48GH27t2LmbF8+XIGDx58wudERKQueT/YzGD/fpg584dwmzrV/3z//hPquR05coSrrrqKl156KTgF05YtW1i0aBHvvPMO6enpREZGBucQ/Pbbb+nUqRP/+c9/+NGPfsS3335Lr169yMjIoF+/fsGVpqdMmcLUqVNZv349L7zwAuPHj6+wjsmTJ/Pzn/+cSy+9lHvvvZfdu3cD/oCLiIggIyMDOHY9NoARI0awePFi1q5dS9euXUtMgCsi0hB5/xe0nYO//c3/eOZM/w/AlCn+9nJmOK+MRo0a0adPHx577DFmBvZbfNkQgEOHDnH22WcD/hncr7nmmuD7Sy9b8vrrrwP+la6Lr6D8zTfflJgUt7TLL7+cnTt3snz5cl577TW6dOnCpk2baN68ebDX1rFjR15++WX++Mc/lnjvyJEjGTVqFFu3bmXMmDFVWlFaRKQ+8n6PDUqGW5ETDDXwL3/y3HPPsX79eu677z6A4LIh6enppKens23bNu4ODHlGR0eXmLy2vGVLCgsLeffdd4P7yM3NpUmTJhXWcuaZZ/LTn/6UBQsW0KNHj+C6ZGPGjOG5555jxYoVJCcnB0O2yLnnnkujRo14/fXXNY2ViHhCeARb0fBjccWvuZ2AmJgYli5dSlpaGo899liFy4ZU1qBBg6q0ltkbb7zBwcDSBfn5+WRlZQVvYomPj6dZs2ZMnz693GVg/vjHP/LAAw+UCF0RkYbK+8FW/JralCn+WZCnTCl5ze0EnXnmmSxfvpw//elPZGZmBpcNSU5OZuDAgezZs6dK+3v44YfZsGEDycnJdOjQgdmzZ1e4/caNG+nevTvJycn07t2b8ePHB4dCwd9r27p1K8OHDy/z/X369OHqq6+uUo0iIvVVeCxbc/fd/htFioYfi8KuadMauTNSJNxo2RqpD8pbtsb7N4+AP7zMfrimVnTN7QSvsYmISP3j/aHIIqVDTKEmIuJJDTrYGsIwqojX6N+d1HcNNtiio6PZt2+f/pGJ1CIzY9++fURHR9d1KSLlarDX2OLi4sjJySEvL6+uSxEJK9HR0SWmcBOpbxpssDVq1IhWrVrVdRkiIlLPNNihSBERkbIo2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU8JWbA559o659KL/XzjnLvdOdfZOfeuc+4j59w/nXOnhaoGEREJPyELNjPbZmY+M/MB3YCDwIvAP4DpZpYUeD4tVDWIiEj4qa2hyP5Alpl9ArQF1gTaXweuqaUaREQkDNRWsI0Gng083gQMCzy+FrigrDc45252zm1wzm3Iy8urhRJFRMQLQh5szrmT8AfZ4kDTTcBk59xGoAnwXVnvM7O5ZtbdzLo3b9481GWKiIhHRNXCMQYD75vZXgAz2woMAnDOJQJX1EINIiISJmpjKHIMPwxD4pw7O/BnBPBbYHYt1CAiImEipMHmnIsBBgJLijWPcc5tB7YCu4EnQlmDiIiEl5AORZrZQaBZqbaZwMxQHldERMKXZh4RERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiUiDMW3aNNq1a0dycjLDhw9n//79wdfuv/9+EhISaNu2Lf/617+C7Rs3biQpKYmEhARuu+02zAyAI0eOMGrUKBISErjooovIzs6u5U8joaJgE5EGY+DAgWzatIkPP/yQxMRE7r//fgA+/vhjFi5cyObNm1m+fDmTJk2ioKAAgIkTJzJ37lwyMzPJzMxk+fLlADz22GOcccYZ7Nixg6lTp/LrX/+6zj6X1CwFm0gYyM7Opn379kyYMIGOHTsyaNAgDh06VOa2O3bsYMCAAXTu3JmuXbuSlZWFmTFt2jQ6depEUlISixYtAmD16tWkpKQwcuRIEhMTmT59OmlpafTs2ZOkpCSysrIAGDduHKmpqfTt25fExESWLl1arc8xaNAgoqKiAOjVqxc5OTkAvPzyy4wePZrGjRvTqlUrEhISWLduHXv27OGbb76hd+/eOOe4/vrreemll4LvueGGGwAYMWIEK1euDPbmpGFTsImEiczMTCZPnszmzZtp2rQpL7zwQpnbjR07lsmTJ5ORkcHatWuJjY1lyZIlpKenk5GRwYoVK5g2bRp79uwBICMjg5kzZ/LRRx+xYMECtm/fzrp16xg/fjyPPPJIcL/Z2dm8+eabLFu2jNTUVA4fPnzMsfv27YvP5zvmZ8WKFcds+/jjjzN48GAAcnNzueCCC4KvxcXFkZubS25uLnFxcce0l35PVFQUp59+Ovv27avqaZV6KKquCxCR2tGqVSt8Ph8A3bp1K/OaUn5+Prm5uQwfPhyA6OhoAN5++23GjBlDZGQk55xzDikpKaxfv57TTjuNHj16EBsbC0B8fDyDBg0CICkpiVWrVgX3PXLkSCIiImjTpg2tW7dm69atwXqKvPXWW5X6LPfeey9RUVGMHTsWoMyelnOu3PaK3iMNn4JNJEw0btw4+DgyMrLMocjyhuIqGqIrvt+IiIjg84iICI4ePRp8rXRolBUiffv2JT8//5j2GTNmMGDAAACefPJJli5dysqVK4P7iIuL47PPPgtun5OTw3nnnUdcXFxwuLJ4e/H3xMXFcfToUb7++mvOPPPMcj+nNBwaihSRoNNOO424uLjgdagjR45w8OBB+vXrx6JFiygoKCAvL481a9bQs2fPKu178eLFFBYWkpWVxc6dO2nbtu0x27z11lukp6cf81MUasuXL+eBBx7glVdeISYmJvi+YcOGsXDhQo4cOcKuXbvIzMykZ8+exMbG0qRJE9577z3MjKeeeoqrrroq+J4nn3wSgOeff57LLrtMPTaPULCJSAkLFizg4YcfJjk5mT59+vD5558zfPhwkpOT6dy5M5dddhkPPvgg5557bpX227ZtW1JSUhg8eDCzZ88mOjqa3bt3M2TIkErv49ZbbyU/P5+BAwfi8/lITU0FoGPHjowcOZIOHTrw4x//mFmzZhEZGQnAo48+yvjx40lISCA+Pj54Xe7nP/85+/btIyEhgb/+9a/8+c9/rtLnkfrLNYS7gLp3724bNmyo6zJEpJrGjRvH0KFDGTFiRF2XIh7inNtoZt1Lt6vHJiIinqKbR0TC1OTJk3nnnXdKtE2ZMoUbb7yxxo81f/78Gt+nSHkUbCJhatasWXVdgkhIaChSREQ8RcEmIjUqLQ1atoSICP+faWl1XZGEGw1FikiNSUuDm2+Ggwf9zz/5xP8cIDBJiEjIqccmIjXmzjt/CLUiBw/620Vqi4JNRGrMp59Wrb2qFi9eTMeOHYmIiKD077ZqPTYpomATkRpz4YVVa6+qTp06sWTJEvr161eiXeuxSXEKNpEwUFvrsSUlTeekk9KAnkASkEVMDFx4Yc2sx9a+ffsy55jUemxSnG4eEQkTmZmZPPvss8ybN4+RI0fywgsv8LOf/eyY7caOHcv06dMZPnw4hw8fprCwsMR6bF988QU9evQI9poyMjLYsmULZ555Jq1bt+aKK8bz/vvr+OSTmTRp8giPPvoQr7/+w3psWVlZXHrppezYsSO4LE6RyszuX5bc3Fx69eoVfF607lqjRo2qvB7bWWedVYWzKvWRgk0kTNTWemy//OUgLr4Y3ngjiYcfXsXYsfD66zW7HltpWo9NilOwiYQJr6zHVhatxybF6RqbiATV9/XYyqP12KQ4BZuIlFCf12N78cUXiYuL49133+WKK67g8ssvB7Qem5Sk9dhEJOS0HpuEgtZjExGRsKCbR0TClNZjE69SsImEKa3HJl6loUgREfEUBZuI1CitxyZ1TUORIlJjtB6b1AfqsYlIjdF6bFIfVNhjc87FAaOBvsB5wCFgE7AMeM3MCkNeoYg0GKFej02kMsrtsTnnngAeB74DHgDGAJOAFcCPgbedc/3Ke7+IhJ9Qr8cmUhkV9dj+18w2ldG+CVjinDsJKPfr6pxrCywq1tQauAtYDcwGooGjwCQzW1fFukWkHrr33pLX2ABiYvztIrWl3B5bWaHmnIt3ziUFXv/OzHZU8P5tZuYzMx/QDTgIvAg8CPwh0H5X4LmIeMDYsTB3LrRoAc75/5w7t+ZuHLn77rs5//zz8fl8+Hw+Xn311eBr999/PwkJCbRt25Z//etfwfaNGzeSlJREQkICt912mxYTDQOVvivSOfcb/EviFjrnCs3suiocpz+QZWafOOcMOC3Qfjqwuwr7EZF6buzY0N4BOXXqVP7nf/6nRNvHH3/MwoUL2bx5M7t372bAgAFs376dyMhIJk6cyNy5c+nVqxdDhgxh+fLlwYmQxZsqusb2C+dcZLGmzmY2xszGAp2reJzRwLOBx7cDf3HOfQbMAP5fOce/2Tm3wTm3IS8vr4qHE5HisrOzad++PRMmTKBjx44MGjSozPXYAHbs2MGAAQPo3LkzXbt2JSsrCzNj2rRpdOrUiaSkJBYt8l9lWL16NSkpKYwcOZLExESmT59OWloaPXv2JCkpiaysLMA/CXJqaip9+/YlMTGRpUuX1ujne/nllxk9ejSNGzemVatWJCQksG7dOvbs2cM333xD7969cc5x/fXXB5fkEe+q6Hb/r4DlzrkrA8//7Zx70zn3FvCvCt5XQuBa3DBgcaBpIjDVzC4ApgKPlfU+M5trZt3NrHvz5s0rezgRKUdmZiaTJ09m8+bNNG3alBdeeKHM7caOHcvkyZPJyMhg7dq1xMbGsmTJEtLT08nIyGDFihVMmzaNPXv2AJCRkcHMmTP56KOPWLBgAdu3b2fdunWMHz+eRx55JLjf7Oxs3nzzTZYtW0ZqaiqHDx8+5th9+/YNDjMW/1mxYkVwm7///e8kJydz00038dVXXwGQm5vLBRdcENwmLi6O3NxccnNziYuLO6ZdvK2ia2xPA1cCPufcy8AGYDAw1MymVeEYg4H3zWxv4PkNwJLA48VA1VYrFJFqadWqFT6fD4Bu3bqRnZ19zDb5+fnk5uYyfPhwAKKjo4mJieHtt99mzJgxREZGcs4555CSksL69esB6NGjB7GxsTRu3Jj4+HgGDRoEQFJSUoljjBw5koiICNq0aUPr1q3ZunXrMcc/3kKjEydOJCsri/T0dGJjY/nlL38JlL3Ct3Ou3HbxtuNdY4vHf2fjPOAewPDf8PF1FY4xhh+GIcF/TS0F/92RlwGZVdiXiFRT48aNg48jIyPLHIos78aKim64KL7fiIiI4POIiAiOHj0afK10oJQVMH379iU/P/+Y9hkzZjBgwADOOeecYNuECRMYOnQo4O+JffbZZ8HXcnJyOO+884iLiyMnJ+eYdvG2iq6xzcd//et+4A4zmwA8Csxzzv2uMjt3zsUAA/mhhwYwAfhf51wGcB9wc/VKF5GadtpppxEXFxe8DnXkyBEOHjxIv379WLRoEQUFBeTl5bFmzRp69qzaYMvixYspLCwkKyuLnTt30rZt22O2OV6PrWj4E/yraXfq1AmAYcOGsXDhQo4cOcKuXbvIzMykZ8+exMbG0qRJE9577z3MjKeeeoqrrrqqmmdHGoqKemxdzKwzgHPuAwAz+wC40jlXqW+GmR0EmpVqexv/7f8iUg8tWLCAW265hbvuuotGjRqxePFihg8fzrvvvkvnzp1xzvHggw9y7rnnljmcWJ62bduSkpLC3r17mT17NtHR0ezevZvx48eXuG2/Ir/61a9IT0/HOUfLli2ZM2cOAB07dmTkyJF06NCBqKgoZs2aRWSk/963Rx99lHHjxnHo0CEGDx6sOyLDgCtviME59wDQCzgJWGJmf6nNworr3r27bdiwoa4OLyInaNy4cQwdOpQRI0bUdSniIc65jWbWvXR7uT02M/u1c+40oNDMDoS0OhERkRpSbrA5534GPFPeRMfOuXggNjC0KCINzOTJk3nnnXdKtE2ZMoUbb7yxxo81f/78Gt+nSHkqusbWDPjAObcR2Ajk4Z/fMQH/XY1fANNDXqGIhMSsWbPqugSRkKhoKHKmc+7v+G/JvxhIxr9szRbgOjPTQhQiIlLvVPh7bGZWALwe+BEREan3tIK2iNSotDRo2RIiIvx/pqXVdUUSbio9u7+IyPGkpZVcj+2TT/zPIbQz/osUd9weW6kZ/kVEynXnnSUXGQX/8zvvrJt6JDxVZihyh3PuL865DiGvRkQatE/LuaWsvHaRUKhMsCUD24F/OOfeC6yTdtrx3iQi4efCC6vWLhIKxw02M8s3s3lm1gf4FfB7YI9z7knnXELIKxSRBuPeeyEmpmRbTIy/XaS2VOoam3NumHPuRWAm8L9Aa+CfQOVmLhWRsDB2LMydCy1agHP+P+fO1Y0jUrsqc1dkJrAK+IuZrS3W/rxzrl9oyhKRhmrsWAWZ1K0Kgy1wR+R8M/tjWa+b2W0hqUpERKSaKhyKDMw8cmkt1SIiInLCKjMUuTYwZ+Qi4NuiRjN7P2RViYiIVFNlgq1P4M/iw5GGf3JkERGReuW4wWZmGooUEZEGo1JzRTrnrgA64l+PDYDybigRERGpS5X5PbbZwCjgF4ADrgVahLguERGRaqnMlFp9zOx64Csz+wPQG7ggtGWJiIhUT2WC7XDgz4POufOA74FWoStJRESk+ipzje2fzrmmwF+A9/HfETkvlEWJiIhU1/FmHokAVprZfuAF59xSINrMvq6N4kRERKrqeDOPFOKf9Ljo+RGFmoiI1GeVucb2b+fcNc45F/JqRERETlBlrrHdAZwCHHXOHcZ/y7+ZmRYbFRGReqcyM480qY1CREREasJxg628NdfMbE3NlyMiInJiKjMUOa3Y42igJ7ARTYIsIiL1UGWGIq8s/tw5dwHwYMgqEhEROQGVuSuytBygU00XIiIiUhMqc43tEfyzjYA/CH1ARghrEhERqbbKXGPbUOzxUeBZM3snRPWIiIickMoE2/PAYTMrAHDORTrnYszsYGhLExERqbrKXGNbCZxc7PnJwIrQlCMiInJiKhNs0WZ2oOhJ4HFM6EoSERGpvsoE27fOua5FT5xz3YBDoStJRESk+ipzje12YLFzbnfgeSwwKmQViYiInIDK/IL2eudcO6At/gmQt5rZ9yGvTEREpBqOOxTpnJsMnGJmm8zsI+BU59yk0JcmIiJSdZW5xjYhsII2AGb2FTAhZBWJiIicgMoEW0TxRUadc5HASaErSUREpPoqc/PIv4DnnHOz8U+tlQosD2lVIiIi1VSZYPs1cDMwEf/NI/8G5oWyKBERkeo67lCkmRWa2WwzG2Fm1wCbgUdCX5qIiEjVVabHhnPOB4zB//tru4AlIaxJRESk2soNNudcIjAaf6DtAxYBzswuraXaREREqqyiHttW4C3gSjPbAeCcm1orVYmIiFRTRdfYrgE+B1Y55+Y55/rjv3lERESk3io32MzsRTMbBbQDVgNTgXOcc4865wbVUn0iIiJVUpm7Ir81szQzGwrEAenA9FAXJiIiUh2VmXkkyMy+NLM5ZnZZqAoSERE5EVUKNhERkfpOwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinRIVqx865tsCiYk2tgbuA3kDbQFtTYL+Z+UJVh4iIhJeQBZuZbQN8AM65SCAXeNHMHiraxjn3v8DXoapBRETCT8iCrZT+QJaZfVLU4JxzwEjgslqqQUREwkBtXWMbDTxbqq0vsNfMMst6g3PuZufcBufchry8vJAXKCIi3hDyYHPOnQQMAxaXemkMx4ZdkJnNNbPuZta9efPmoSxRREQ8pDaGIgcD75vZ3qIG51wU8BOgWy0cX0REwkhtDEWW1TMbAGw1s5xaOL6IiISRkAabcy4GGAgsKfVSWdfcRERETlhIhyLN7CDQrIz2caE8roiIhC/NPCIiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeIqCTUREPEXBJiINxrRp02jXrh3JyckMHz6c/fv3A5Cdnc3JJ5+Mz+fD5/ORmpoafM/GjRtJSkoiISGB2267DTOro+qltijYRKTBGDhwIJs2beLDDz8kMTGR+++/P/hafHw86enppKenM3v27GD7xIkTmTt3LpmZmWRmZrJ8+fK6KF1qkYJNJAxkZ2fTvn17JkyYQMeOHRk0aBCHDh0qc9sdO3YwYMAAOnfuTNeuXcnKysLMmDZtGp06dSIpKYlFixYBsHr1alJSUhg5ciSJiYlMnz6dtLQ0evbsSVJSEllZWQCMGzeO1NRU+vbtS2JiIkuXLq3W5xg0aBBRUVEA9OrVi5ycnAq337NnD9988w29e/fGOcf111/PSy+9VK1jS8OhYBMJE5mZmUyePJnNmzfTtGlTXnjhhTK3Gzt2LJMnTyYjI4O1a9cSGxvLkiVLSE9PJyMjgxUrVjBt2jT27NkDQEZGBjNnzuSjjz5iwYIFbN++nXXr1jF+/HgeeeSR4H6zs7N58803WbZsGampqRw+fPiYY/ft2zc4nFj8Z8WKFcds+/jjjzN48ODg8127dtGlSxdSUlJ46623AMjNzSUuLi64TVxcHLm5udU7gdJgRNV1ASJSO1q1aoXP5wOgW7duZGdnH7NNfn4+ubm5DB8+HIDo6GgA3n77bcaMGUNkZCTnnHMOKSkprF+/ntNOO40ePXoQGxsL+IcDBw0aBEBSUhKrVq0K7nvkyJFERETQpk0bWrduzdatW4P1FCkKpOO59957iYqKYuzYsQDExsby6aef0qxZMzZu3MjVV1/N5s2by7ye5pyr1DGk4VKwiYSJxo0bBx9HRkaWORRZ3o0VFd1wUXy/ERERwecREREcPXo0+FrpQCkrYPr27Ut+fv4x7TNmzGDAgAEAPPnkkyxdupSVK1cG99G4cePgcbt160Z8fDzbt28nLi6uxHBlTk4O5513XrmfRbxBQ5EiEnTaaacRFxcXvA515MgRDh48SL9+/Vi0aBEFBQXk5eWxZs0aevbsWaV9L168mMLCQrKysti5cydt27Y9Zpu33noreANI8Z+iUFu+fDkPPPAAr7zyCjExMcH35eXlUVBQAMDOnTvJzMykdevWxMbG0qRJE9577z3MjKeeeoqrrrqqmmdHGgoFm4iUsGDBAh5++GGSk5Pp06cPn3/+OcOHDyc5OZnOnTtz2WWX8eCDD3LuuedWab9t27YlJSWFwYMHM3v2bKKjo9m9ezdDhgyp9D5uvfVW8vPzGThwYInb+tesWROsb8SIEcyePZszzzwTgEcffZTx48eTkJBAfHx8iety4k2uIfxOR/fu3W3Dhg11XYaIVNO4ceMYOnQoI0aMqOtSxEOccxvNrHvpdvXYRETEU3TziEiYmjx5Mu+8806JtilTpnDjjTfW+LHmz59f4/sUKY+CTSRMzZo1q65LEAkJDUWKiIinKNhEpEZNmgRRUeCc/89Jk+q6Igk3GooUkRozaRI8+ugPzwsKfnj+f/9XNzVJ+FGPTURqzNy5VWsXCQUFm4jUmMDkH5VuFwkFBZuI1JjIyKq1i4RCyILNOdfWOZde7Ocb59ztgdd+4Zzb5pzb7Jx7MFQ1iEjtuvnmqrWLhELIbh4xs22AD8A5FwnkAi865y4FrgKSzeyIc+7sUNUgIrWr6AaRuXP9w4+Rkf5Q040jUptq667I/kCWmX3inPsL8GczOwJgZv+tpRpEpBb83/8pyKRu1dY1ttHAs4HHiUBf59x/nHNvOud6lPUG59zNzrkNzrkNeXl5tVSmiIg0dCEPNufcScAwYHGgKQo4A+gFTAOec2WsOGhmc82su5l1b968eajLFBERj6iNHttg4H0z2xt4ngMsMb91QCFwVi3UISIiYaA2gm0MPwxDArwEXAbgnEsETgK+qIU6REQkDIQ02JxzMcBAYEmx5seB1s65TcBC4AZrCKudiohIgxDSuyLN7CDQrFTbd8DPQnlcEREJX5p5REREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKeomATERFPUbCJiIinKNhERMRTFGwiIuIpCjYREfEUBZuIiHiKgk1ERDxFwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIi4ikKNhER8ZSwCbY9+XuIfziezw98XteliIhICIVNsN2z5h6y92dzz5v31HUpIiISQmERbHvy9/BE+hMUWiFPpD+hXpuIiIeFRbDds+YeCq0QgAIrUK9NRMTDPB9sRb217wq+A+C7gu/UaxMR8TDPB1vx3loR9dpERLzL88H2yrZXgr21It8VfMfL216uo4pERCSUouq6gFDLuSOnrksQEZFa5Pkem4iIhBcFm4iIeIqCTUREPEXBJiIinqJgExERT1GwiYiIpyjYRETEUxRsIiLiKQo2ERHxFAWbiIh4ioJNREQ8RcEmIiKe4sysrms4LudcHvBJNd56FvBFDZfjdTpn1aPzVnU6Z9Wj8/aDFmbWvHRjgwi26nLObTCz7nVdR0Oic1Y9Om9Vp3NWPTpvx6ehSBER8RQFm4iIeIrXg21uXRfQAOmcVY/OW9XpnFWPzttxePoam4iIhB+v99hERCTMKNhERMRTGnywOed+7Jzb5pzb4ZybXsbrzjn3cOD1D51zXeuizvqmEuftEufc18659MDPXXVRZ33inHvcOfdf59ymcl7Xd62USpwzfc/K4Jy7wDm3yjm3xTm32Tk3pYxt9H0rj5k12B8gEsgCWgMnARlAh1LbDAFeAxzQC/hPXddd1z+VPG+XAEvrutb69AP0A7oCm8p5Xd+1qp8zfc/KPi+xQNfA4ybAdv23rfI/Db3H1hPYYWY7zew7YCFwValtrgKeMr/3gKbOudjaLrSeqcx5k1LMbA3wZQWb6LtWSiXOmZTBzPaY2fuBx/nAFuD8Upvp+1aOhh5s5wOfFXuew7F/+ZXZJtxU9pz0ds5lOOdec851rJ3SGjR916pH37MKOOdaAl2A/5R6Sd+3ckTVdQEnyJXRVvr3FyqzTbipzDl5H/88bAecc0OAl4A2oS6sgdN3rer0PauAc+5U4AXgdjP7pvTLZbxF3zcafo8tB7ig2PM4YHc1tgk3xz0nZvaNmR0IPH4VaOScO6v2SmyQ9F2rIn3Pyueca4Q/1NLMbEkZm+j7Vo6GHmzrgTbOuVbOuZOA0cArpbZ5Bbg+cAdRL+BrM9tT24XWM8c9b865c51zLvC4J/7vyr5ar7Rh0XetivQ9K1vgnDwGbDGzv5azmb5v5WjQQ5FmdtQ5dyvwL/x3+j1uZpudc6mB12cDr+K/e2gHcBC4sa7qrS8qed5GABOdc0eBQ8BoC9yKFa6cc8/iv4vvLOdcDvB7oBHou1aeSpwzfc/KdjFwHfCRcy490PYb4ELQ9+14NKWWiIh4SkMfihQRESlBwSYiIp6iYBMREU9RsImIiKco2ERExFMUbCIBzrlmxWaZ/9w5l1vs+UnHeW9359zDlTjG2hqq1ReYqaPo+bCyVmmooWM95JzrV8HrtzrndKu51Bu63V+kDM65u4EDZjajWFuUmR2tu6p+4JwbB3Q3s1tDfJwzgVfNrFcF28QA75hZl1DWIlJZ6rGJVMA5N98591fn3CrgAedcT+fcWufcB4E/2wa2u8Q5tzTw+O7AOmSrnXM7nXO3FdvfgWLbr3bOPe+c2+qcSys2A8eQQNvbgfW2lpaq6STgj8CoQG9ylHNunHPu78VqftT51/Pa6ZxLCdSzxTk3v9h+Bjnn3nXOve+cWxyYl7C0EcDyYu/5s3PuY+df/2sGgJkdBLIDM4eI1LkGPfOISC1JBAaYWYFz7jSgX2D2lgHAfcA1ZbynHXAp/rW0tjnnHjWz70tt0wXoiH9+v3eAi51zG4A5gWPsCszcUYKZfef8C3IGe2yBHlxxZwCXAcOAf+KfyWI8sN4558M/z+BvA5/rW+fcr4E78AdmcRcDzweOcSYwHGhnZuaca1psuw1AX2BdGedCpFYp2ESOb7GZFQQenw486Zxrg38m9UblvGeZmR0Bjjjn/gucgz9MiltnZjkAgWmTWgIHgJ1mtiuwzbPAzdWo+Z+B8PkI2GtmHwWOszlwnDigA/BOoKN4EvBuGfuJBfICj78BDgP/cM4tA4r3JP+LP8xF6pyCTeT4vi32+B5glZkNd/51slaX854jxR4XUPa/tbK2KWspkuoo2ndhqeMUBo5TALxuZmOOs59DQDQE5xjtCfTHP3H2rfh7hQS2OVQzpYucGF1jE6ma04HcwONxIdj/VqB1IDQBRpWzXT7+Yc7qeg//0GcC+G8Acc4llrHdFqBom1OB0wPLy9wO+IptlwhsOoF6RGqMgk2kah4E7nfOvYN/ZYQaZWaHgEnAcufc28Be4OsyNl0FdCi6eaQax8nDH8zPOuc+xB90ZQ0lLsM/Oz/4g3RpYPs3ganFtrsYWFHVOkRCQbf7i9QzzrlTAytKO2AWkGlmf6vDet4GhprZ/nJe7wLcYWbX1WphIuVQj02k/pkQuJlkM/6hzzl1Ww6/JLAOWDnOAn5XS7WIHJd6bCIi4inqsYmIiKco2ERExFMUbCIi4ikKNhER8RQFm4iIeMr/B5xHyEtAAna2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ray\n",
    "import codeflare.pipelines.Datamodel as dm\n",
    "import codeflare.pipelines.Runtime as rt\n",
    "from codeflare.pipelines.Datamodel import Xy\n",
    "from codeflare.pipelines.Datamodel import XYRef\n",
    "from codeflare.pipelines.Runtime import ExecutionType\n",
    "\n",
    "ray.shutdown()\n",
    "ray.init()\n",
    "\n",
    "n_runs = 3\n",
    "score_250 = 0\n",
    "score_500 = 0\n",
    "score_1000 = 0\n",
    "score_2000 = 0\n",
    "run_time = 0\n",
    "\n",
    "for _ in range(n_runs):\n",
    "    \n",
    "    start = time.time()\n",
    "    \n",
    "    pipeline = dm.Pipeline()\n",
    "    node_pcs250 = dm.EstimatorNode(\"kernel_approximator_250\", PolynomialCountSketch(n_components=250,\n",
    "                                        degree=4))\n",
    "    node_pcs500 = dm.EstimatorNode(\"kernel_approximator_500\", PolynomialCountSketch(n_components=500,\n",
    "                                        degree=4))\n",
    "    node_pcs1000 = dm.EstimatorNode(\"kernel_approximator_1000\", PolynomialCountSketch(n_components=1000,\n",
    "                                        degree=4))\n",
    "    node_pcs2000 = dm.EstimatorNode(\"kernel_approximator_2000\", PolynomialCountSketch(n_components=2000,\n",
    "                                        degree=4))\n",
    "    node_clf250 = dm.EstimatorNode(\"linear_classifier_250\", LinearSVC())\n",
    "    node_clf500 = dm.EstimatorNode(\"linear_classifier_500\", LinearSVC())\n",
    "    node_clf1000 = dm.EstimatorNode(\"linear_classifier_1000\", LinearSVC())\n",
    "    node_clf2000 = dm.EstimatorNode(\"linear_classifier_2000\", LinearSVC())\n",
    "    \n",
    "    pipeline.add_edge(node_pcs250, node_clf250)\n",
    "    pipeline.add_edge(node_pcs500, node_clf500)\n",
    "    pipeline.add_edge(node_pcs1000, node_clf1000)\n",
    "    pipeline.add_edge(node_pcs2000, node_clf2000)\n",
    "    \n",
    "    # create training input\n",
    "    train_input = dm.PipelineInput()\n",
    "    xy_train = dm.Xy(X_train, y_train)\n",
    "    train_input.add_xy_arg(node_pcs250, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs500, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs1000, xy_train)\n",
    "    train_input.add_xy_arg(node_pcs2000, xy_train)\n",
    "    \n",
    "    # codeflare pipeline is able to execute model FIT once with four n_components \n",
    "    pipeline_fitted = rt.execute_pipeline(pipeline, ExecutionType.FIT, train_input)\n",
    "    \n",
    "    xy_test = dm.Xy(X_test, y_test)\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs250, xy_test)\n",
    "    pipeline_250 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf250)[0])\n",
    "    score_250 += 100 * ray.get(rt.execute_pipeline(pipeline_250, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf250)[0].get_Xref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs500, xy_test)\n",
    "    pipeline_500 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf500)[0])\n",
    "    score_500 += 100 * ray.get(rt.execute_pipeline(pipeline_500, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf500)[0].get_Xref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs1000, xy_test)\n",
    "    pipeline_1000 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf1000)[0])\n",
    "    score_1000 += 100 * ray.get(rt.execute_pipeline(pipeline_1000, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf1000)[0].get_Xref())\n",
    "    \n",
    "    test_input = dm.PipelineInput()\n",
    "    test_input.add_xy_arg(node_pcs2000, xy_test)\n",
    "    pipeline_2000 = rt.select_pipeline(pipeline_fitted, pipeline_fitted.get_xyrefs(node_clf2000)[0])\n",
    "    score_2000 += 100 * ray.get(rt.execute_pipeline(pipeline_2000, ExecutionType.SCORE, test_input)\n",
    "                    .get_xyrefs(node_clf2000)[0].get_Xref())\n",
    "\n",
    "    run_time += ((time.time() - start)/4)\n",
    "\n",
    "# all runs have the the same run time since they are FITTED together\n",
    "results[f\"LSVM + PS(250)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_250/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(500)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_500/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(1000)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_1000/n_runs\n",
    "}\n",
    "results[f\"LSVM + PS(2000)\"] = {\n",
    "        \"time\": run_time/n_runs, \"score\": score_2000/n_runs\n",
    "}\n",
    "\n",
    "\n",
    "N_COMPONENTS = [250, 500, 1000, 2000]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 7))\n",
    "ax.scatter([results[\"LSVM\"][\"time\"], ], [results[\"LSVM\"][\"score\"], ],\n",
    "           label=\"Linear SVM\", c=\"green\", marker=\"^\")\n",
    "\n",
    "ax.scatter([results[\"LSVM + PS(250)\"][\"time\"], ],\n",
    "           [results[\"LSVM + PS(250)\"][\"score\"], ],\n",
    "           label=\"Linear SVM + PolynomialCountSketch\", c=\"blue\")\n",
    "for n_components in N_COMPONENTS:\n",
    "    ax.scatter([results[f\"LSVM + PS({n_components})\"][\"time\"], ],\n",
    "               [results[f\"LSVM + PS({n_components})\"][\"score\"], ],\n",
    "               c=\"blue\")\n",
    "    ax.annotate(f\"n_comp.={n_components}\",\n",
    "                (results[f\"LSVM + PS({n_components})\"][\"time\"],\n",
    "                 results[f\"LSVM + PS({n_components})\"][\"score\"]),\n",
    "                xytext=(-30, 10), textcoords=\"offset pixels\")\n",
    "\n",
    "ax.scatter([results[\"KSVM\"][\"time\"], ], [results[\"KSVM\"][\"score\"], ],\n",
    "           label=\"Kernel SVM\", c=\"red\", marker=\"x\")\n",
    "\n",
    "ax.set_xlabel(\"Training time (s)\")\n",
    "ax.set_ylabel(\"Accurary (%)\")\n",
    "ax.legend()\n",
    "plt.show()\n",
    "\n",
    "ray.shutdown()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
